Maintaining two-site configuration for workload availability between sites at unlimited distances for products and services

ABSTRACT

A system for maintaining a two-site configuration for continuous availability over long distances may include a first computing site configured to execute a first instance associated with a priority workload, the first instance being designated as an active instance; a second computing site configured to execute a second instance of the priority workload, the second instance being designated as a standby instance; a software replication module configured to replicate a unit of work data associated with the priority workload from a first data object associated with the active instance to a second data object associated with the standby instance, and a hardware replication module configured to replicate an image from a first storage volume to a copy on a second storage volume, wherein the first storage volume is associated with the first computing site, and the second storage volume is associated with a third computing site.

DOMESTIC PRIORITY

This application is a continuation of and claims priority from U.S.patent application Ser. No. 14/059,686, filed on Oct. 22, 2013, entitled“MAINTAINING TWO-SITE CONFIGURATION FOR WORKLOAD AVAILABILITY BETWEENSITES AT UNLIMITED DISTANCES FOR PRODUCTS AND SERVICES”, the entirecontents of which are incorporated herein by reference.

BACKGROUND

The present invention relates to continuous availability between sitesat unlimited distances, and more specifically, to maintaining a two-siteconfiguration in a multisite continuous availability computingenvironment.

In the past, some computer availability and disaster recovery solutionswere limited to a maximum distance between sites. Other past solutionsrequired starting systems, applications, and supporting infrastructureon the backup site that could in some cases take several hours torestart. Some past solutions additionally required modifications tosoftware applications, such as database servers, and hardware, such asrouters and switches, in order to implement various disaster recoveryand continuous availability functions, resulting in relatively highimplementation cost. Some past solutions operated at a site level,rather than at a workload level.

These issues have been substantially addressed by continuousavailability solutions between sites at unlimited distances. However,when a primary or secondary continuous availability computer processingsite becomes unavailable, for example, due to an unplanned outage orduring planned maintenance at one of the sites, the remaining site mayrun in a single-site configuration, without continuous workloadavailability, throughout the duration of the primary or secondary siteunavailability, until such time as the site becomes available onceagain. This scenario may expose some workloads to a higher level of riskduring a substantial period of time.

In addition, some solutions provide continuous workload availabilityonly with regard to routable, online transaction processing (OLTP)applications and related data objects. Non-OLTP workloads, or “batch”workloads, may not be supported. In some cases, nonsupported workloadsmay target the same data objects as supported workloads. However, aperiod of unavailability of a primary site may result in the loss orunavailability of data related to nonsupported workloads, such asintermediate data files and job scheduler states. In addition, datarelated to nonsupported workloads may be inconsistent with duplicatedata related to supported workloads.

SUMMARY

According to one embodiment of the present invention, a system formaintaining a two-site configuration for continuous availability overlong distances includes a first computing site configured to execute afirst instance associated with a priority workload, wherein the firstinstance is designated as an active instance, a second computing siteconfigured to execute a second instance of the priority workload,wherein the second instance is designated as a standby instance, and aworkload availability module configured to replicate a unit of work dataassociated with the priority workload from a first data objectassociated with the active instance to a second data object associatedwith the standby instance, and to replicate an image from a firststorage volume to a copy on a second storage volume, wherein the firststorage volume is associated with the first computing site, and thesecond storage volume is associated with a third computing site.

According to another embodiment of the present invention, a computerprogram product for maintaining a two-site configuration for continuousavailability over long distances includes a computer readable storagemedium having stored thereon first program instructions executable by aprocessor to cause the processor to assign a first instance of apriority workload to a first computing site, wherein the first instanceis designated as an active instance, second program instructionsexecutable by the processor to cause the processor to assign a secondinstance of the priority workload to a second computing site, whereinthe second instance is designated as a standby instance, third programinstructions executable by the processor to cause the processor toreplicate a unit of work data associated with the priority workload froma first data object associated with the active instance to a second dataobject associated with the standby instance, and fourth programinstructions executable by the processor to cause the processor toreplicate an image from a first storage volume to a copy on a secondstorage volume, wherein the first storage volume is associated with thefirst computing site, and the second storage volume is associated with athird computing site.

According to yet another embodiment of the present invention, a methodfor maintaining a two-site configuration for continuous availabilityover long distances includes executing a first instance associated witha priority workload at least in part with a first processor, wherein thefirst instance is designated as an active instance, executing a secondinstance associated with the priority workload at least in part with asecond processor, wherein the second instance is designated as a standbyinstance, replicating a unit of work data associated with the priorityworkload from a first data object associated with the active instance toa second data object associated with the standby instance, andreplicating an image from a first storage volume to a copy on a secondstorage volume, wherein the first storage volume is associated with thefirst processor, and the second storage volume is associated with athird processor.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention. For a better understanding of the invention with theadvantages and the features, refer to the description and to thedrawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The forgoing and other features, and advantages ofthe invention are apparent from the following detailed description takenin conjunction with the accompanying drawings in which:

FIG. 1 depicts a cloud computing node according to an embodiment of thepresent invention.

FIG. 2 depicts a cloud computing environment according to an embodimentof the present invention.

FIG. 3 depicts abstraction model layers according to an embodiment ofthe present invention.

FIG. 4 illustrates a schematic diagram of an integralcontinuous/reliable availability system in accordance with an embodimentof the present invention.

FIG. 5 illustrates a schematic diagram of the various components in anintegral continuous/reliable availability system in accordance with anembodiment of the present invention.

FIG. 6A is a schematic diagram that illustrates a multisite,multiworkload configuration of an integral continuous/reliableavailability system in accordance with an embodiment of the presentinvention.

FIG. 6B is a schematic diagram that illustrates another multisite,multiworkload configuration of an integral continuous/reliableavailability system in accordance with an embodiment of the presentinvention.

FIG. 6C is a schematic diagram that illustrates yet another multisite,multiworkload configuration of an integral continuous/reliableavailability system in accordance with an embodiment of the presentinvention.

FIG. 6D is a schematic diagram that illustrates yet another multisite,multiworkload configuration of an integral continuous/reliableavailability system in accordance with an embodiment of the presentinvention.

FIG. 6E is a schematic diagram that illustrates yet another multisite,multiworkload configuration of an integral continuous/reliableavailability system in accordance with an embodiment of the presentinvention.

FIG. 7 illustrates a schematic diagram of an individual siteimplementation of an integral continuous/reliable availability system inaccordance with an embodiment of the present invention.

FIG. 8 illustrates a process flow for providing integralcontinuous/reliable availability in accordance with an embodiment of thepresent invention.

FIG. 9 illustrates another process flow for providing integralcontinuous/reliable availability in accordance with an embodiment of thepresent invention.

FIG. 10 illustrates a process flow for a failover procedure inaccordance with an embodiment of the present invention.

FIG. 11 illustrates a process flow for a restore procedure in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as Follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as Follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as Follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 1, a schematic of an example of a cloud computingnode is shown. Cloud computing node 10 is only one example of a suitablecloud computing node and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the inventiondescribed herein. Regardless, cloud computing node 10 is capable ofbeing implemented and/or performing any of the functionality set forthhereinabove.

In cloud computing node 10 there is a computer system/server 12, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 1, computer system/server 12 in cloud computing node 10is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 12 may include, but are not limitedto, one or more processors or processing units 16, a system memory 26,and a bus 18 that couples various system components including systemmemory 26 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnect (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 26 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 28 and/or cachememory 30. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 32 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 26 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 34, having a set (at least one) of program modules 36,may be stored in memory 26 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 36 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Referring now to FIG. 2, illustrative cloud computing environment 38 isdepicted. As shown, cloud computing environment 38 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 40A, desktop computer 40B, laptop computer 40C,and/or automobile computer system 40N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 38 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 40A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 38 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 3, a set of functional abstraction layers providedby cloud computing environment 38 (FIG. 2) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 3 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 42 includes hardware and softwarecomponents. Examples of hardware components include mainframes, in oneexample IBM® zSeries® systems; RISC (Reduced Instruction Set Computer)architecture based servers, in one example IBM pSeries® systems; IBMxSeries® systems; IBM BladeCenter® systems; storage devices; networksand networking components. Examples of software components includenetwork application server software, in one example IBM WebSphere®application server software; and database software, in one example IBMDB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter,WebSphere, and DB2 are trademarks of International Business MachinesCorporation registered in many jurisdictions worldwide).

Virtualization layer 44 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers;virtual storage; virtual networks, including virtual private networks;virtual applications and operating systems; and virtual clients.

In one example, management layer 46 may provide the functions describedbelow. Resource provisioning provides dynamic procurement of computingresources and other resources that are utilized to perform tasks withinthe cloud computing environment. Metering and Pricing provide costtracking as resources are utilized within the cloud computingenvironment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal provides access to the cloud computing environment forconsumers and system administrators. Service level management providescloud computing resource allocation and management such that requiredservice levels are met. Service Level Agreement (SLA) planning andfulfillment provide pre-arrangement for, and procurement of, cloudcomputing resources for which a future requirement is anticipated inaccordance with an SLA.

Workloads layer 48 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation; software development and lifecycle management; virtualclassroom education delivery; data analytics processing; transactionprocessing; and mobile desktop.

With regard to the resource provisioning and service level managementfunctions of the management layer 46, an embodiment of the presentinvention may provide continuous availability of priority workloads,reliable availability of general workloads, disaster recovery, workloaddistribution and replication of application data across a plurality ofsites at unlimited distances.

Some existing availability systems are limited geographically and/or byrecovery time. When one or more workloads are spread across multipleservers in a single location, the servers for each workload may share asingle data repository, and all data related to each of workloads may bestored in the same location. When the workloads are split amonggeographically dispersed data centers, a single data repository for eachworkload is not always feasible.

In these instances, data from the one or more workloads may be stored ina data repository at a primary site, and the data may be synchronized,bit-by-bit, between the primary site and a copy of the data at thesecondary site. The time that it takes to synchronize the databases iscalled latency. As sites are spread further apart geographically,latency may increase because of the time it takes to move the data overa network in order to synchronize it. Once latency increases beyond arelatively small amount of time, transferring data between data centersrequires increasingly longer periods of time to achieve synchronization.

As a result, some existing availability systems provide acceptableworkload performance only within a limited geographic area. In somecases, this limited geographic area may be approximately 10 to 20 fiberkilometers (i.e., 10 to 20 linear kilometers of a fiber optic network).

Disaster recovery systems are designed to switch between a primary datacenter and a backup data center in situations where the primary datacenter becomes unavailable, such as, for example, during a power outage.For example, during normal operation all transactions may be distributedto the primary data center and the data may be periodically replicatedbit-by-bit to the secondary site or sites.

Priority workloads generally may be executed in parallel on at least twodistinct computing systems. Typically, at least two instances of apriority workload may be executed virtually simultaneously on at leasttwo geographically dispersed computing systems, for example, an activeinstance executing on a computing system at a primary site and a standbyinstance executing on another computing system at a secondary site. Sucha configuration may sometimes be referred to in the art as an“active/active” workload. General workloads, such as batch applicationworkloads, workloads that do not log data, workloads that use sequentialfiles, flat files, or other non-logged data objects, or the like, on theother hand, typically may be executed on only one computing system at asingle location, for example, an active instance executing on acomputing system at a primary site. Such a configuration may sometimesbe referred to in the art as a “non-active/active” workload.

Some clients require continuous availability for priority workloads andreliable availability for general workloads. In addition, some clientsrequire that disaster recovery systems be performed at a workload level.Furthermore, some clients require that availability system sites beseparated by relatively long distances. Thus, an integrated availabilitysolution is required that may provide continuous availability at aworkload level for priority workloads, as well as reliable availabilityfor general workloads, between multiple sites separated by relativelylong distances.

In an embodiment of the present invention, an integratedcontinuous/reliable availability system may include a softwarereplication module and a hardware replication module. The softwarereplication module may provide unit of work-based software replicationmethods for one or more priority workloads, and the hardware replicationmodule may provide storage unit-based, managed hardware replicationmethods for one or more storage units across multiple sites separated byrelatively long distances. The integrated continuous/reliableavailability system may not require modification to existing userapplications.

In any embodiment of the present invention, the distance between sitesmay include, for example, distances greater than the area covered withina metro area network (MAN), that is, a network that may span distancesmeasured in tens of kilometers, for example, up to about 20 fiberkilometers. Some customers require that a primary site and a secondaryredirection site be separated by distances sufficient to ensure that adisaster affecting one site is not likely to affect the other. Althoughthese distances vary based on regional and environmental conditions,primary and secondary sites sometimes are separated by distances thatextend beyond a MAN. In various embodiments, the distance between sitesmay include, for example, distances greater than 50 fiber kilometers,distances greater than 100 fiber kilometers, or distances greater than1,000 fiber kilometers.

In some embodiments, the customer acceptability window may be measuredby the length of the RPO. For example, in an embodiment, the customeracceptable window requires an RPO of less than 3 seconds of data losswhen an unplanned interruption occurs.

In any embodiment of the invention, the software replication methods maysupport continuous availability, workload distribution, replication andservices for multiple priority workloads executing on multiple computingsystems at multiple individual site locations separated by an unlimitedgeographic area, and may provide minimal downtime or nearlyinstantaneous priority workload redirection, such as, for example,within less than 3 seconds, at a common point in time consistency forpriority workload data. In combination, the hardware replication methodsmay support reliable availability, workload distribution and replicationservices for multiple general workloads executing on multiple computingsystems at multiple individual site locations separated by an unlimitedgeographic area, and may provide minimal-downtime data synchronizationand workload redirections, such as, for example, within less than onehour, for general workload data.

A workload may consist of one or more computing applications or jobs, aswell as associated middleware runtime environments, data source objectsused by the applications, and the network addressability of theapplications. A priority workload may consist of one or more computingapplications, jobs or threads that are relatively time-sensitive andpreferably will not be suspended for more than a brief moment, such as,for example, less than five seconds, even in the event of an emergencyor unplanned failure. A general workload may consist of one or morecomputing applications, jobs or threads that are less time-sensitive andmay be suspended for more than a brief moment, and preferably as long asapproximately one hour, in the event of an emergency or unplannedfailure.

A unit of work may include one or more computing transactions and/orprocesses substantially performed as a group to service one or morerequests. A unit of work data may include, for example, data generatedby or otherwise associated with a single computing transaction and/orprocess, or with multiple computing transactions and/or processessubstantially performed as a group to service one or more requests. Adata object may include, for example, any combination of related orassociated data.

In an embodiment, the integrated continuous/reliable availability systemmay include a workload distribution module that collects metrics at thesoftware application, middleware, operating system, network, andhardware levels for each workload. The integrated continuous/reliableavailability system and may use the collected metrics to providecontinuous availability and workload redirection capabilities acrossmultiple computing sites.

With reference now to FIG. 4, an embodiment of the present invention mayinclude an integrated continuous/reliable availability system 50 forimplementing continuous availability for priority workloads and reliableavailability for general workloads across multiple sites at unlimiteddistances. The system 50 may include a workload distribution module 52executing computer instructions. The workload distribution module 52 mayoperate in any type of environment that is capable of executing asoftware application. For example, the workload distribution module 52may include a high-speed, multiuser, multitasking computer processingdevice, such as a mainframe computer. In some embodiments, the workloaddistribution module 52 may be associated with an enterprise (e.g., acommercial business) that implements the integrated continuous/reliableavailability across multiple sites at unlimited distances.

The integrated continuous/reliable availability depicted in FIG. 4 mayinclude one or more computing sites, such as, for example, site one 54,site two 56 and site three 69. Each of the sites 54, 56, 69 may includeone or more systems executing one or more workloads. The workloads mayinclude transaction processing applications, database applications,queue and queue management operations, and the like. Each of the sites54, 56, 69 may include, for example, one or more network hardwaredevices and/or software for managing and distributing network traffic.

Site one 54, site two 56 and site three 69 may be geographicallydistributed computing sites. For example, site one 54 may be located inone region, for example Region A 67, and site two 56 and site three 69may be located in another region, for example, Region B 68, that isrelatively geographically distant from Region A 67. The geographicdistance between Region A 67 and Region B 68 may provide for arelatively high probability that computer processing sites in Region A67 will not suffer outages, or otherwise become unavailable, at the sametime as computer processing sites in Region B 68. In particular, thegeographic distance between Region A 67 and Region B 68 may provide fora relatively high probability that computer processing sites in Region A67 and sites in Region B 68 will not suffer outages, or otherwise becomeunavailable, due to a common cause, such as a regional power outage. Inan alternative embodiment, site three 69 may be located in a thirdregion that is relatively geographically distant from Region A 67 andfrom Region B 68.

The integrated continuous/reliable availability system 50 depicted inFIG. 4 additionally may include a software replication module 58 and ahardware replication module 60. The software replication module 58,which will be described in more detail below, may replicate data forpriority workloads between site one 54 and site two 56. The hardwarereplication module 60, which also will be described in more detailbelow, may replicate data for storage units, including priority workloaddata and general workload data, such as disk replication, between siteone 54 and site three 69. The integrated continuous/reliableavailability system 50 further may include a controller 62, which maycontrol the operation of the various components of the integratedcontinuous/reliable availability system 50, including, for example, theworkload distribution module 52, which is described in more detailbelow.

The workload distribution module 52 and the sites 54, 56, 69 may becommunicatively coupled via one or more networks 64. The networks 64 maybe implemented using any type or combination of known networking device,including, but not limited to, a wide area network (WAN), a local areanetwork (LAN), a global network (e.g., Internet), a virtual privatenetwork (VPN), an intranet and a telephone network. The networks 64 maybe implemented using a wireless network or any kind of physical networkimplementation known in the art.

The sites, such as site one 54, site two 56 and site three 69 may becoupled to the workload distribution module 52 through multiple networks(e.g., intranet and Internet) such that not all of the sites are coupledto the workload distribution module 52 through the same network. Theworkload distribution module 52 may be implemented using one or moreservers, for example, operating in response to a computer program storedin a storage medium accessible by the server.

In the integrated continuous/reliable availability system 50, units ofwork 66 initiated by users of the various systems or clients executingat the one or more sites may be distributed to one or more of the sites54, 56, 69 through the workload distribution module 52. The units ofwork 66 may be transmitted from systems outside of the sites 54, 56, 69and may be processed as workloads within one or more of the sites.

It will be readily understood by a person of ordinary skill in the artthat the execution of continuous/reliable availability across multiplesites at unlimited distances system and methods described in FIG. 4 maybe implemented as modules in hardware, software executing ongeneral-purpose hardware, or a combination thereof. Although only threesites are depicted in FIG. 4, it will be further understood that, in anembodiment, any number of sites may be implemented, and that anygeographic distance may separate the sites. Furthermore, although theworkload distribution module 52 is depicted as existing outside of thesites, it will be readily understood by a person of ordinary skill inthe art that, in an embodiment, the workload distribution module 52 maybe directly located at one or more of the sites.

FIG. 5 illustrates a schematic diagram of the various components inaccordance with another embodiment of the invention. An integratedcontinuous/reliable availability system 70 includes a workloaddistribution module 72. In an embodiment, the workload distributionmodule 72 may collect metrics from multiple computing sites, forexample, site two 76 and either site one 74 or site three 106. Themetrics collected for each of the workloads may include, for example,processor speed, pending transactions, transaction execution time,system availability, network bandwidth utilization and availability, andany other performance-based metrics known in the art. The workloaddistribution module 72 may use the metrics in order to distribute one ormore units of work 78 for one or more workloads to site one 74 and sitetwo 76.

Individual units of work may be received or may be initiated at one ofthe site one 74 or site two 76. For example, in some embodiments siteone 74 may include a computer system that is simultaneously orintermittently executing one or more priority workloads 80 and one ormore general workloads 90. In other embodiments, site one 74 may includea group of servers, such as a server farm, operating on one or moreworkloads 80, 90 using local load balancing, or other methods of loaddistributing as is known in the art. In yet another embodiment, site one74 may include multiple systems, each of which may execute one or moreworkloads 80, 90. In various embodiments, site one 74 may include acombination of servers and server farms each operating on one or moreworkloads.

In addition, site one 74 may include one or more monitoring modules,such as site one monitoring module 82. The site one monitoring module 82may be communicatively coupled to the workload distribution module 72,such as through a network, and may transmit metrics from the site one 74to the workload distribution module 72. In some embodiments, the siteone monitoring module 82 may be executed on a single computer. In otherembodiments, a monitoring module is executed on each of the systemsexecuting at the site one 74. In yet other embodiments, multiplemonitoring modules, one on each server, monitor and report metrics tothe workload distribution module 72.

Furthermore, the site one monitoring module 82 may be configured tomonitor the systems executing at site one 74. In some embodiments, thesite one monitoring module 82 may be configured to monitor the availablehardware processing capacity of the computer processors executing at thesite one 74. In other embodiments, the site one monitoring module 82 maybe configured to monitor the available network capacity of the site one74. In yet other embodiments, the site one monitoring module 82 may beconfigured to monitor the one or more workloads 80, 90 executing at thesite one 74.

In various embodiments, the site one monitoring module 82 may monitorvarious characteristics of the workloads 80, 90 such as the number ofqueued transactions, the availability of the workloads 80, 90 to handleadditional transactions, the number of threads associated with each ofthe one or more workloads 80, 90, and any other workload-specificcharacteristics as is known in the art. Similarly, site two 76 mayinclude a site two monitoring module 86, the operation of which may beanalogous to that of the site one monitoring module 82.

In addition, site one 74 may include a software replication module 88, ahardware replication module 98, and a storage unit 94. Likewise, sitetwo 76 may include a software replication module 100, a hardwarereplication module 102, and a storage unit 104. An additional storageunit 96 may be located at a third site, for example, site three 106.

The software replication modules 88, 100 may be configured to repeatedlyor continuously replicate units of work, or unit of work data, from thepriority workloads 80, 84 from the respective sites 74, 76. The softwarereplication modules 88, 100 may collect units of work from the priorityworkloads 80, 84 and coordinate the replication of those units of workon the other site 76, 74 at relatively frequent intervals or periods,such as, for example, at a substantially real-time rate, in order tomaintain the active and standby instances of a workload substantiallysynchronized. For example, the software replication modules 88, 100 mayreplicate unit of work data, such as logged transactional data, that is,logged data based on transactional boundaries, from the primary site,for example, site one 74, to the secondary site, for example, site two76, subsequent to each transaction at the primary site. In variousembodiments, the software replication modules 88, 100 may implement anysuitable method of duplicating a workload instance, including, forexample, program instructions, associated data and state information,such as IBM's InfoSphere Data Replication method.

The hardware replication modules 98, 102 may be configured toperiodically replicate the contents of the storage units 94, 104 of therespective sites 74, 76, including priority workload data and generalworkload data. The hardware replication modules 98, 102 collect contentfrom the storage units 94, 104 and coordinate the periodic replicationof those contents on site three 106 at relatively less frequentintervals or periods, such as, for example, once every 5 seconds, onceevery 10 seconds, once every 30 seconds or once per minute. In anembodiment, the hardware replication modules 98, 102 may replicatecontents of the storage units 94, 104 simultaneously with or immediatelyafter each write function to the storage units 94, 104. In anembodiment, the hardware replication modules 98, 102 perform diskreplication, copying or mirroring a complete image of a storage unit.

In various embodiments, the hardware replication modules 98, 102 mayimplement any suitable method of duplicating or mirroring a storagevolume, image, or contents to a remote site, such as IBM's Metro Mirrorpeer-to-peer remote copy (PPRC), extended remote copy (XRC), or GlobalMirror disk replication architectures, or the like. In variousembodiments, the periodic hardware replication may be synchronous orasynchronous.

Multiple workloads may execute on separate sites, and each may bereplicated to one or more other sites. For example, a priority workload80 may execute on site one 74 and be replicated to site two 76, whileanother priority workload 84 executes on site two 76 and issimultaneously replicated on site one 74. In an embodiment, if themetrics for each workload indicate that one of the sites is overloaded,the workload distribution module 72 may distribute all units of work forthat workload to another site. Of course, in various embodiments, anynumber of additional sites may be configured to provide load balancingand replication of units of work.

Although the controller 108 of FIG. 5 is depicted as a stand-alonemodule, it will be understood that, in various embodiments, thecontroller 108 may be executed in the workload distribution module 72 orin any combination of the sites 74, 76, 106. For example, in anembodiment, the controller 108 may communicate with each of the sites74, 76, 106 and may be configured to coordinate transactions andreplication of the units of work between the various sites. Thecontroller 108 may communicate with the workload distribution module 72,and use information provided by each of those modules to coordinatetransactions and replication of the units of work for each workloadbetween the various sites.

The illustration of FIG. 5 is a simplified representation of the variouscomponents of the integrated continuous/reliable availability system 70for purposes of clarity. It will be understood by those of ordinaryskill in the art, that additional or fewer components may be used inalternate embodiments. In additional embodiments, the layout andconfiguration of the components may differ from those of FIG. 5 withoutaffecting the functionality of the integrated continuous/reliableavailability system 70. In additional embodiments, the variouscomponents may be located in separate modules. In further embodiments,the functionality of various components may be incorporated into asingle hardware or software module.

FIG. 6A is a schematic diagram that illustrates a simplified multisite,multiworkload, integrated continuous/reliable availability system 110,including multiple geographically distributed computing sites, forexample, Site A 112, Site B 114 and Site C 116. Site A 112 may belocated in one region, for example Region 1 (not shown), and Site 114and Site C 116 may be located in another region, for example, Region 2(not shown), that is relatively geographically distant from Region 1.The geographic distance between Region 1 and Region 2 may provide for arelatively high probability that computer processing sites in Region 1will not suffer outages, or otherwise become unavailable, at the sametime as computer processing sites in Region 2. In particular, thegeographic distance between Region 1 and Region 2 may provide for arelatively high probability that computer processing sites in Region 1and sites in Region 2 will not suffer outages, or otherwise becomeunavailable, due to a common cause, such as a regional power outage.

In any embodiment, Site C 116 may be located in a third region, forexample, Region 3 (not shown), that also is relatively geographicallydistant from Region 1. As above, the geographical distance betweenRegion 1 and Region 3 may provide for a relatively high probability thatcomputer processing sites in Region 1 and computer processing sites inRegion 3 will not become unavailable at the same time or due to a commoncause.

The primary computing site in this example, Site A 112, may host anactive priority workload 118 and an active general workload 120. That isto say, the active priority workload 118 and an active general workload120 may be assigned to Site A. The active priority workload 118 may beprovided with continuous availability and workload redirectionprovisions, including software replication methods, because it is apriority workload. For example, a standby priority workload 122 may beexecuted in parallel on Site B 114. That is, the program instructions,associated data and state information of active priority workload 118may be replicated from Site A 112 to Site B, for example, by thesoftware replication module 58 of FIG. 4, and the priority workloadprogram instructions may be simultaneously executed at Site A 112 and atSite B 114 as an active priority workload 118 and as a standby priorityworkload 122.

In addition, the active general workload 120 may be provided withreliable availability and workload redirection provisions, includingmanaged hardware replication techniques, because it is a generalworkload. For example, the contents, or image, of a storage unit 126 atSite A (Image A), including the program instructions, may beperiodically replicated to maintain a copy of the contents on a storageunit 132 at Site C (Copy A). The storage units 126, 128, 132 may includeany type of computer memory medium organized in any format, such as, forexample, a relational model database server, a hierarchical database, aninformation management system, a virtual storage access method server, ahard disk drive (HDD), optical storage medium, magnetic tape, or anyother acceptable memory medium. A database may include, for example, anygroup of files organized in association with any database manager knownin the art.

The software replication and hardware replication may be coordinated, ormanaged, by workload availability module 124, which may incorporate thefunctionality of the workload distribution module 52, the softwarereplication module 58 and the hardware replication module 60 of FIG. 4,and may be communicatively linked with a controller 130. In anyembodiment, the workload availability module 124 may communicate withthe various sites via a network, such as the one or more networks 64 ofFIG. 4. Thus, at any given moment in time, the contents of the storageunit 126 at Site A 112 may be backed up by a mirrored copy at a backupsite, such as the storage unit 132 at Site C 116, which may be availablein the case that Site A 112 should become unavailable.

The workload availability module 124 may be configured to detect thatSite A 112 is unavailable, or that the active priority workload 118 isnot executing on the primary site. In this case, as shown in FIG. 6B,workload availability module 124 may reassign and automatically redirectactive priority workload 118 (i.e., transmit the ongoing/future datastream of active priority workload 118) to Site B 114, and designate theexecution of the priority workload on Site B as the active priorityworkload 118. In various embodiments the redirection may occur, forexample, within about 10 seconds, 30 seconds, one minute or 3 minutes.In various embodiments, the redirection of priority workloads may befully automated, may require authorization by an operator, for example,based upon an operational policy, or may require initiation by anoperator, for example, in the case of planned maintenance.

In any embodiment, workload redirection may occur because of anemergency or unplanned system or site outage, for example, based onmetrics received from Site A 112. Alternatively, in an embodiment, theworkload redirection may occur because of a planned system or siteoutage, for example, initiated by a program script and/or instructionsfrom an operator.

In order to provide substantially continuous or near continuoussecondary backup of the priority workload, the workload availabilitymodule 124 may restart the hardware replicated image at anotheravailable site, for example, bringing up Copy A from the storage unit132 at site C 116. Thus, the workload availability module 124 may assignand replicate the active priority workload 118, for example, as standbypriority workload 122 on Site C 116, as shown in FIG. 6C. Thus, duringthe time that Site A 112 is unavailable, the priority workload maycontinue to execute in parallel as an active priority workload 118 onSite B 114 and as a standby priority workload 122 on Site C 116.

At this point the contents of the storage unit 132 at Site C 116 (ImageC) may actively support the standby priority workload 122 running atSite C 116, while continuing to maintain a residual copy of all datarelated to general workloads that previously were running at Site A 112before Site A 112 became unavailable. Thus, the only period during whichan active backup of priority workloads is not provided is the brief timerequired to replicate the active priority workload 118 from the originalsecondary site, Site B 114, to the new secondary site, Site C 116.

When a priority workload, such as active priority workload 118, isredirected from Site A 112 to Site B 114, the active general workload120 may continue to execute on site A 112 unimpeded, if Site A 112continues to be available. However, should workload availability module124 detect that Site A 112 is unavailable, or that the active generalworkload 120 is not executing on the primary site, as shown in FIG. 6C,the workload availability module 124 may reassign the general workloads,for example, to Site C 116, and restart the active general workload 120at Site C 116 from the hardware copy maintained in storage unit 132 atSite C 116.

The workload availability module 124 may synchronize data related topriority and general workloads in storage unit 132 at Site C 116 withthat of storage unit 128 at Site B 114, for example, implementingsoftware replication methods, to resolve any data inconsistencies suchthat the standby priority workloads and general workloads may becomeresynchronized with the active priority workloads, such as activepriority workload 118, at disk integration. For example, the workloadavailability module 124 may resolve a data inconsistency by replacingdata in storage 132 at Site C 116 related to a priority workload, suchas priority workload 118, with corresponding data in storage 128 at SiteB 114 based on the data in storage unit 132 at Site C 116 having becomeobsolete due to updates performed at Site B 116 but not at Site C 116over time, for example, the period of time required to restart theactive general workload 120 at Site C 116 from the hardware copy.

In various embodiments the redirection and restart of priority andgeneral workloads from the hardware copy may occur, for example, withinabout two hours, within about one hour, or within about one-half hour.In various embodiments, the redirection of general workloads may befully automated, may require authorization by an operator for example,based upon an operational policy, or may require initiation by anoperator, for example, in the case of planned maintenance.

In addition, the workload availability module 124 may initiatereplication of the contents (Image C) of storage unit 132 to anotheravailable site, for example, on storage unit 126 once Site A 112 becomesavailable again, as shown in FIG. 6D, in order to facilitate theeventual return to the original site configuration. In this scenario,the only period during which a hardware backup of general workloadswould not be provided is the time during which Site A 112 is notavailable plus the time required to replicate the storage unit 132contents (Image C) from the original hardware backup site, Site C 116,to the storage unit 126 at Site A 112 (Copy C).

In any embodiment, during the time that Site A 112 is unavailable, thecontents (Image C) of storage unit 132 may also be periodicallyreplicated at yet another site (not shown) in Region 1, Region 2, oranother region that is relatively geographically distant from Region 3.The periodic hardware replication of storage unit 132 contents mayprovide reliable availability and facilitate the eventual return to theoriginal site configuration. Thus, the only period during which ahardware backup of general workloads may not be provided is the timerequired to replicate the storage unit 132 contents (Image C) from theoriginal hardware backup site, Site C 116, to a storage unit at a newsecondary hardware backup site.

In FIG. 6D, Site B 114 has become the primary site for priorityworkloads, and Site C 116 has become the primary site for generalworkloads, as well as the secondary, or backup, site for priorityworkloads. The operative configuration of Site B 114 and Site C 116 inFIG. 6D may provide the full functionality, including software andhardware replication, of the original operative configuration of Site A112 and Site B 114 in FIG. 6A. The active priority workload 118 maycontinue to execute indefinitely on Site B, and the active generalworkload 120 and the standby priority workload 122 may continue toexecute indefinitely on Site C 116, while the image of storage unit 132may be indefinitely periodically copied to storage unit 126 at Site A112, or to a storage unit at yet another site while Site A isunavailable.

Nevertheless, after Site A 112 becomes available once again, forexample, after maintenance or repairs have been performed, power hasbeen restored, or the like, then the workload availability module 124may reverse the migration process, or “go home,” and return theoperative configuration to the original operational configuration, suchas, for example, a normal operation mode. The workload availabilitymodule 124 may replicate the contents (Image C) of the storage unit 132at Site C 116 to the storage unit 126 at Site A 112 (Copy C) as shown inFIG. 6D.

As depicted in FIG. 6E, execution of workloads at the third site may behalted, and the workload availability module 124 may restart thepriority workload as standby priority workload 122 at Site A 112. Atthis point the contents of the storage unit 126 at Site A 112 (Image A)may actively support the standby priority workload 122 running at Site A112, while continuing to maintain a residual copy of all data related togeneral workloads running at Site C 116.

The workload availability module 124 may reassign and automaticallyredirect active priority workload 118 (i.e., transmit the ongoing/futuredata stream of active priority workload 118) to Site A 112, and onceagain designate the execution of the priority workload on Site A 112 asthe active priority workload 118, returning to the original siteconfiguration of FIG. 6A. Substantially simultaneously, workloadavailability module 124 may designate the execution of the priorityworkload on Site B 114 as the standby priority workload 122. In variousembodiments the redirection may occur, for example, within about 10seconds, 30 seconds, one minute or 3 minutes.

The workload availability module 124 may reassign the general workloadsto Site A 112 and restart the active general workload 120 at Site A 112from the hardware copy in storage unit 126 at Site A 112, as shown inFIG. 6A. The workload availability module 124 may synchronize datarelated to priority and general workloads in storage unit 132 at Site A112 with that of storage unit 128 at Site B 114, for example,implementing software replication methods, to resolve any datainconsistencies such that the standby priority workloads and generalworkloads may become resynchronized with the active priority workloads,such as active priority workload 118, at disk integration. For example,the workload availability module 124 may resolve a data inconsistency byreplacing data in storage 126 at Site A 112 related to a priorityworkload, such as the active priority workload 118, with correspondingdata in storage 128 at Site B 114 based on the data in storage unit 126at Site A 112 having become obsolete due to updates performed at Site B116 but not at Site A 112 over time, for example, the period of timerequired to restart the active general workload 120 at Site A 112 fromthe hardware copy.

The contents (Image A) of the storage unit 126 at Site A 112, may onceagain be periodically replicated to maintain a copy of the contents(Copy A), for example, on the storage unit 132 at Site C 116. At thispoint, Site A 112, Site B 114 and Site C 116 may all have been returnedto the original operative configuration of FIG. 6A. In variousembodiments, the reverse migration, that is, the redirection and restartof priority and general workloads from the hardware copy, may requireapproximately three hours, approximately one hour, or approximately 30minutes.

In various embodiments, the workload availability module 124 may performthe reverse migration, or “go home,” procedures based on automatedprogramming scripts and/or user instructions. For example, in anembodiment, the workload availability module 124 may restore theoriginal operative configuration upon detecting that the primary site isonce again available. In an embodiment, the reverse migration must beinitiated by a user, or operator.

The operative configurations of the priority and general workloads,sites and storage units of FIG. 6A-6E are provided for purposes ofclarity, and a person of ordinary skill in the art will readilyunderstand that in alternate embodiments any number of sites andworkloads may be implemented in any combination, and that the sequenceof operational configuration transitions may be performed in a varietyof different orders to accomplish continuous/reliable availability.Similarly, a person of ordinary skill in the art will readily apprehendthat a similar sequence of operational configuration transitions may beperformed in the case of a secondary site unavailability, such as anoutage of Site B 114 of FIG. 6A.

FIG. 7 illustrates a block diagram of an individual site 140 associatedwith an integral continuous/reliable workload availability module 142 inaccordance with an embodiment of the invention. The workloadavailability module 142 may incorporate the functionality of theworkload distribution module 52, the software replication module 58 andthe hardware replication module 60 of FIG. 4, and may be communicativelycoupled to a controller 164 and to one or more workloads executing atthe site 140. The workload availability module 142 may coordinatedistribution of units of work for the active priority workload 144. Eachthe active priority workload 144 and the active general workload 146 mayinclude an application interface 148, 150 that may facilitatecommunication of units of work to the active priority workload 144, theactive general workload 146, or both.

The application interfaces 148, 150 may be configured to use any type ofapplication interface known in the art, such as, for example, TCP/IP,message queuing, remote procedure execution, or any other suitableinterface. Each the active priority workload 144 and the active generalworkload 146 additionally may include a transaction and data storageunit 152, 154. In one embodiment, the transaction and data storage units152, 154 may include, for example, a database storage system. In anotherembodiment, the transaction and data storage units 152, 154 may includea file-based system. In yet another embodiment, the transaction and datastorage units 152, 154 may include a transaction-based storage such as aqueue. In other embodiments, the transaction and data storage units 152,154 may be any storage as is known in the art.

The active priority workload 144 additionally may be associated with aworkload monitoring module 156. In The workload monitoring module 156may monitor the performance of the workloads and the system processingload. The workload monitoring module 156 may be configured to determinethe transaction processing speed of the workloads, the number of threadsexecuting for each workload, the number of transactions queued forprocessing, and/or any other workload processing related information.The workload monitoring module 156 may be communicatively coupled to amonitoring module, such as the site one monitoring module 82 of FIG. 5,which may transmit the workload metrics to the workload availabilitymodule 142.

The active priority workload 144 and the active general workload 146 mayfurther include system state monitors 160, 162. The system statemonitors 160, 162 may communicate to the workload availability module142 whether or not the active priority workload 144 and the activegeneral workload 146 are currently operating within specifiedtolerances. When either the active priority workload 144 or the activegeneral workload 146 should stop operating correctly, the system statemonitors 160, 162 may notify the workload availability module 142.

FIG. 8 illustrates a process flow for providing integralcontinuous/reliable availability in accordance with an embodiment of theinvention. At block 170, statistical data and metrics may be collectedfor each workload from the various applications and sites. In anembodiment, the statistical data and metrics are collected continuouslyat a substantially real-time rate. At block 172, a unit of work may bereceived for a workload. The unit of work may be associated with eithera priority workload or with a general workload. A unit of work mayinclude one or more transactions that are connected with one another. Inan embodiment, the unit of work includes a series of updates and/orinserts in a relational database and the unit of work is defined by afirst transaction, and terminated by a commit request, which closes thegroup of transactions and stores them in a database. In anotherembodiment, a logical unit of work may include a series of transactionsacross multiple backend systems, each dependent on one another.

At block 174, a workload distribution module, such as workloaddistribution module 72 of FIG. 5, may use the network addressing for theunit of work to determine with which workload the unit of work isassociated. At block 176, the particular site that will be used toexecute the unit of work may be determined and the unit of work may beassigned to the site based on the workload determined at block 174, andthe metrics collected at block 170.

In an embodiment, the site may be selected based on whether or not thesite is the primary site for executing the workload that the unit ofwork is directed to, on the available processing capacity, the availablenetwork bandwidth, the average transaction execution speed, the numberof pending transactions, the availability of a site, and/or any otherfactor as is known in the art. In another embodiment, the site may beselected iteratively in a “round-robin” fashion. In yet anotherembodiment, the site may be selected randomly from a group of availablesites.

In an embodiment, if one site is unavailable, such as where a networkbecomes unavailable, a power outage is encountered or a hardware failureexists, the site may be automatically removed from consideration untilthe issues have been corrected. In an additional embodiment, if theworkload is unavailable, for example, because of a system error or ascheduled outage, the workload is directed to an alternate site that iscapable of processing the workload.

At block 178, the unit of work may be transmitted to the selected site.At block 180, a system within the site may be selected to process thework. In some embodiments, a plurality of systems at each site may beconfigured to process a unit of work. At block 182, the unit of work maybe transmitted to the selected system. At block 184, the unit of workmay be executed by the selected system at the selected site, and thetransactions in the unit of work may be committed and stored in adatabase.

At block 186, once the transactions have been executed, the unit of workdata may be replicated, for example, if the unit of work is associatedwith a priority workload, to the other site that supports the sameworkload environment using a software replication module such as each ofthe software replication modules 88, 100 of FIG. 5. In variousembodiments, the replication may be synchronous or asynchronous. In anadditional embodiment, the unit of work data may be replicated in itsentirety to multiple alternate sites using software replication methods.

At block 188 each of the transactions in the unit of work may becommitted to each of the alternative sites, for example, by a softwarereplication module, and executed at each of the alternate sites in a waythat maintains data consistency. Technical effects and benefits ofsoftware replication include a mechanism for load balancing, workloadredirection, and replication of data associated with one or moreworkload across a number of sites separated by unlimited distanceswithout requiring workload changes.

At block 190, the content, image or data maintained in a storage unit,such as, for example, a hard disk drive, at the primary site may bereplicated, or mirrored, in its entirety, or substantially in itsentirety, for example, to a storage unit at a third site that isgeographically distant from the primary site. In an alternativeembodiment, the data may be replicated from the primary site to morethan one other site. In another embodiment, the data copy at the thirdsite may be replicated to one or more other sites.

FIG. 9 illustrates an additional process flow for providing integralcontinuous/reliable availability in accordance with an embodiment of theinvention. At block 192, metrics or statistical data regarding theprimary computing site may be monitored to verify correct operation ofthe primary site. At block 194, a determination may be made as towhether or not the primary site is available. If not, in block 196 afailover procedure may be performed, for example, according to FIG. 10.If the primary site is available, the active priority workloads may beassigned to and executed at the primary site in block 198.

In block 200, the standby priority workloads may be executed at thesecondary site, and in block 202 priority workload data may bereplicated, for example, using software replication techniques from theprimary site to the secondary site. The active general workloads may beassigned to and executed at the primary site in block 204, and in block206, the storage unit data maintained in a storage unit at the primarysite may be replicated to a storage unit at a third site, for example,using hardware replication techniques.

FIG. 10 illustrates a failover procedure that may be used with theprocess flow for providing integral continuous/reliable availability ofFIG. 9. In the event that the primary site is not available, in block210, the active priority workloads may be switched over to execute atthe secondary site. For example, the standby instances of priorityworkloads executing at the secondary site may be designated as andbecome the primary, or active, instances of the priority workloads. Theswitchover may be requested by a user, operator or system administrator,or automated according to a policy implemented, for example, inprogramming script. In various embodiments, the switchover may requireapproximately 10 seconds, 30 seconds, one minute or three minutes.

In block 212, the standby priority workloads may be reassigned to andrestarted at a third site from a hardware copy, implementing softwarereplication methods with the second site in order to maintain continuousavailability for the priority workloads. For example, the priorityworkloads previously executing at the primary site may be restarted atthe third site as standby instances using dormant instances copied tothe mirrored storage volume at the third site by a hardware replicationmethod before the primary site became unavailable. The restart may berequested by a user, operator or system administrator, or automatedaccording to a policy implemented, for example, in programming script.

In block 214, the active general workloads may be restarted at the thirdsite. For example, the general workloads previously executing at theprimary site may be reassigned to and restarted at the third site usingdormant instances copied to the mirrored storage volume at the thirdsite by a managed hardware replication method before the primary sitebecame unavailable. The restart may be requested by a user, operator orsystem administrator, or automated according to a policy implemented,for example, in programming script. In various embodiments, the restartof priority and general workloads may require approximately three hours,approximately one hour, or approximately 30 minutes.

In block 216, data related to priority and general workloads at thethird site may be synchronized with the current active priority workloaddata, for example, implementing software replication methods. Forexample, data in a storage unit at the third site used by priority orgeneral workloads may be synchronized with corresponding data in astorage unit at the original secondary site to resolve any datainconsistencies, for example, resulting from data at the third sitehaving become obsolete due to updates performed at the originalsecondary site but not at the third site during the period of timerequired to restart the priority and general workloads from the hardwarecopy at the third site, such that the priority and general workloads atthe third site may become resynchronized with the priority workloads atthe original secondary site. The data used by general workloadsgenerally may include, for example, batch application files, sequentialfiles, flat files, state information, other files that do not log data,non-logged data objects, or the like.

A determination may be made, in block 218, as to whether or not theprimary site is currently available. If the primary site remainsunavailable, the process may wait at block 218 until the primary sitebecomes available once again. In block 220, the contents or data copymaintained in a storage unit at the third site may be replicated to theoriginal primary site using hardware replication methods.

At block 222, an operator input may be monitored for a “restore”command. At block 224, a determination may be made as to whether or notthe restore command has been received. If the restore command has beenreceived, at block 226, a restore procedure may be performed, forexample, according to FIG. 11. However, if the restore command has notbeen received in block 224, the procedure may return to block 218.

A person of ordinary skill in the art will readily understand that thismethod reduces the potential disruption of computing services, andreduces the exposure time during which priority workloads may operate ata single site without continuous backup resulting from a failure to thetime required to restart workloads from the mirrored storage volume.

FIG. 11 illustrates a restore, or reverse migration, procedure that maybe used with the process flow for providing integral continuous/reliableavailability of FIG. 9 and the failover procedure of FIG. 10. In block230, data related to priority and general workloads at the originalprimary site may be synchronized with the current active priorityworkload data from the secondary site, for example, implementingsoftware replication methods. For example, data in a storage unit at theoriginal primary site used by priority or general workloads may besynchronized with corresponding data in a storage unit at the secondarysite to resolve any data inconsistencies, for example, resulting fromdata at the original primary site having become obsolete due to updatesperformed at the secondary site but not at the original primary siteduring the period of time required to restart the priority and generalworkloads from the hardware copy at the original primary site, such thatthe priority and general workloads at the original primary site maybecome resynchronized with the active priority workloads at thesecondary site. The data used by general workloads generally mayinclude, for example, batch application files, sequential files, flatfiles, state information, other files that do not log data, non-loggeddata objects, or the like.

The standby priority workloads may be reassigned to and restarted at theprimary site from a hardware copy, in block 232, implementing softwarereplication methods with the second site in order to maintain continuousavailability for the priority workloads. For example, execution ofworkloads at the third site may be halted, and the primary workloadsthat were executing at the third site may be restarted at the originalprimary site using dormant instances copied to the mirrored storagevolume at the primary site from the third site by a hardware replicationmethod. In an embodiment, the restart must be requested by a user,operator or system administrator, or automated according to a policyimplemented, for example, in programming script.

The general workloads may be restarted at the original primary site inblock 234. For example, the general workloads executing at the thirdsite may be restarted at the primary site using dormant instances copiedto the mirrored storage volume at the primary site by a hardwarereplication method. In a preferred embodiment, the restore proceduremust be commanded by a user, operator or system administrator. Invarious embodiments, the restart of priority and general workloads atthe original primary site may require approximately three hours,approximately one hour, or approximately 30 minutes.

In block 236, the active priority workloads and the standby priorityworkloads may be switched, or swapped. For example, the standbyinstances of priority workloads that were restarted at the primary sitein block 230 may be designated and become the primary, or active,instances of the workloads, returning primary execution of the priorityworkloads to the primary site, and the priority workloads executing atthe original secondary site may once again be designated as standbyinstances. This switchover may be performed under planned, controlledconditions at a time selected to be convenient to the operator. In apreferred embodiment, the restore procedure must be commanded by a user,operator or system administrator. In various embodiments, the switchovermay require approximately 10 seconds, 30 seconds, one minute or 3minutes.

In block 238, the content or data maintained in a storage unit at theoriginal primary site may be periodically replicated to another storageunit, for example, at the third site using hardware replication methods.

A person of ordinary skill in the art will readily understand that themethods of FIGS. 9-11 reduce the exposure time during which priorityworkloads may operate at a single site without continuous backup duringthe restore procedure to the time required to restart workloads from themirrored storage volume at the primary site.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s).

It should also be noted that, in some alternative implementations, thefunctions noted in the block may occur out of the order noted in thefigures. For example, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved. It will also be noted that each block of the block diagramsand/or flowchart illustration, and combinations of blocks in the blockdiagrams and/or flowchart illustration, can be implemented by specialpurpose hardware-based systems that perform the specified functions oracts, or combinations of special purpose hardware and computerinstructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of onemore other features, integers, steps, operations, element components,and/or groups thereof.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. The corresponding structures, materials, acts,and equivalents of all means or step plus function elements in theclaims below are intended to include any structure, material, or act forperforming the function in combination with other claimed elements asspecifically claimed. The description of the present invention has beenpresented for purposes of illustration and description, but is notintended to be exhaustive or limited to the invention in the formdisclosed. Many modifications and variations will be apparent to thoseof ordinary skill in the art without departing from the scope and spiritof the invention. The embodiment was chosen and described in order tobest explain the principles of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand the invention for various embodiments with variousmodifications as are suited to the particular use contemplated

The flow diagrams depicted herein are just one example. There may bemany variations to this diagram or the steps (or operations) describedtherein without departing from the spirit of the invention. Forinstance, the steps may be performed in a differing order or steps maybe added, deleted or modified. All of these variations are considered apart of the claimed invention.

While the preferred embodiment to the invention has been described, itwill be understood that those skilled in the art, both now and in thefuture, may make various improvements and enhancements which fall withinthe scope of the claims which follow. These claims should be construedto maintain the proper protection for the invention first described.

What is claimed is:
 1. A computer program product for maintaining amultiple-site configuration for continuous availability over longdistances, the computer program product comprising one or morenon-transitory computer-readable storage media having stored thereon:first program instructions executable by a first processor located at afirst computing site, the first program instructions being configured toexecute a first instance of one or more computing transactionsassociated with a priority workload, wherein the first instance isdesignated as an active instance; second program instructions executableby a second processor that is communicatively coupled to the firstprocessor and located at a second computing site, the second programinstructions being configured to execute a second instance of the one ormore computing transactions, wherein the second instance is designatedas a standby instance; and a workload availability module configured todetermine based on monitoring one or more metrics that the firstcomputing site is unavailable and, in response to a determination thatthe first computing site is unavailable, to: designate the secondinstance as the active instance; cause a third computing site to executea third instance of the one or more computing transactions associatedwith the priority workload; and designate the third instance as thestandby instance.
 2. The computer program product of claim 1, whereinthe first program instructions are further configured to restart thefirst instance associated with the priority workload based on the firstcomputing site subsequently becoming available.
 3. The computer programproduct of claim 2, wherein the first program instructions are furtherconfigured to designate the first instance as the active instance afterrestarting the first instance.
 4. The computer program product of claim1, wherein the second computing site is separated from the firstcomputing site by a first distance greater than a metropolitan areanetwork.
 5. The computer program product of claim 1, wherein the secondstorage volume is separated from the first storage volume by a seconddistance greater than a metropolitan area network.
 6. The computerprogram product of claim 1, wherein execution of the third instance isbased at least in part on a copy of data from the first computing sitethat is stored on a storage volume of the second computing site.
 7. Thecomputer program product of claim 1 wherein the non-transitorycomputer-readable storage media further include a first replicationmodule configured to replicate a unit of work data associated with thepriority workload to the standby instance at the second computing site,and wherein the unit of work data includes transactional data that islogged in response to completion of at least one of the one or morecomputing transactions.
 8. A system for maintaining a multiple-siteconfiguration for continuous availability over long distances, thesystem comprising: a first computing site configured to execute a firstinstance of one or more computing transactions associated with apriority workload, wherein the first instance is designated as an activeinstance; a second computing site communicatively coupled with the firstcomputing site, the second computing site configured to execute a secondinstance of the one or more computing transactions, wherein the secondinstance is designated as a standby instance; and a workloadavailability module configured to determine based on monitoring one ormore metrics that the first computing site is unavailable and, inresponse to a determination that the first computing site isunavailable, to: designate the second instance as the active instance;cause a third computing site to execute a third instance of the one ormore computing transactions associated with the priority workload; anddesignate the third instance as the standby instance.
 9. The system ofclaim 8, wherein the first computing site is further configured torestart the first instance associated with the priority workload basedon the first computing site subsequently becoming available.
 10. Thesystem of claim 9, wherein the first computing site is furtherconfigured to designate the first instance as the active instance afterrestarting the first instance.
 11. The system of claim 8, wherein thesecond computing site is separated from the first computing site by adistance greater than a metropolitan area network.
 12. The system ofclaim 8, further comprising a replication module configured to replicatedata stored in a first storage volume associated with the firstcomputing site to a copy on a second storage volume associated with thesecond computing site, wherein the second storage volume is separatedfrom the first storage volume by a distance greater than a metropolitanarea network.
 13. The system of claim 12, wherein execution of the thirdinstance is based at least in part on the copy on the second storagevolume.
 14. The system of claim 8, further comprising a replicationmodule configured to replicate a unit of work data associated with thepriority workload to the standby instance at the second computing site,wherein the unit of work data includes transactional data that is loggedin response to completion of at least one of the one or more computingtransactions.
 15. A computer-implemented method for maintaining amultiple-site configuration for continuous availability over longdistances, the method comprising: executing, at a first computing site,a first instance of one or more computing transactions associated with apriority workload, wherein the first instance is designated as an activeinstance; executing, at a second computing site, a second instance ofthe one or more computing transactions associated with the priorityworkload, wherein the second instance is designated as a standbyinstance; in response to determining, based on monitoring a metricassociated with the first computing site, that the first computing siteis unavailable: designating the second instance as the active instance;causing a third computing site to execute a third instance of the one ormore computing transactions associated with the priority workload; anddesignating the third instance as the standby instance.
 16. Thecomputer-implemented method of claim 15, wherein the first computingsite is further configured to restart the first instance associated withthe priority workload based on the first computing site subsequentlybecoming available.
 17. The computer-implemented method of claim 16,wherein the first computing site is further configured to designate thefirst instance as the active instance after restarting the firstinstance.
 18. The computer-implemented method of claim 15, wherein thesecond computing site is separated from the first computing site by afirst distance greater than a metropolitan area network.
 19. Thecomputer-implemented method of claim 15, wherein execution of the thirdinstance is based at least in part on a copy of data associated with thefirst instance that is stored on a storage volume of the secondcomputing site.
 20. The computer-implemented method of claim 15, furthercomprising replicating a unit of work data associated with the priorityworkload being executed as the active instance at the first computingsite to the standby instance at the second computing site, wherein theunit of work data includes transactional data that is logged in responseto completion of at least one of the one or more computing transactions.